Audio routing system for routing audio data to and from a mobile device

ABSTRACT

A method includes receiving sound by a first audio unit installed in an electrical outlet, routing audio data corresponding to the received sound from the first audio unit to a second audio unit installed in a second electrical outlet, and sending the audio data to a mobile device using a wireless link between the mobile device and the second audio unit. Routing the audio data may include receiving the audio data from the first audio unit by a third audio unit and routing the audio data to the second audio unit by the third audio unit serving as a router. The data may be routed using table driven routing, on-demand routing or some other appropriate routing protocol. The method may also include performing voice recognition on the audio data and detecting a command word and routing command word data to the second audio unit.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to mobile devices and speakerphone extension devices and more particularly to voice recognitionsystems and routing audio data between devices.

BACKGROUND

Many current mobile devices include voice command capability to enableresponse to a user's voice commands for control of voice operateddialing or other features. In some mobile devices the voice commandsystem may be capable of waking the mobile device from a low-power sleepmode. This may be accomplished in such existing mobile devices, byhaving the voice command system active even during the sleep mode andalways listening for a specific command word or command phrase. When anappropriate command word or command phrase is recognized, the voicecommand system wakes the mobile device from sleep mode and appropriatelyresponds to the command. Thus such mobile devices allow for hands-freeinteraction through voice control.

One challenge for mobile device voice command systems is in obtaining aclear audio input to the mobile device microphone or microphone array.For example, if the user is too far from the mobile device, or if themobile device is in a pocket, purse or wedged between sofa cushions suchthat the microphones are obstructed, the user's voice command willeither not be picked up by the microphones or will not be recognizableby the voice command system.

One way to overcome this challenge is for the user to wear a separate,remote microphone ensemble such as a headset or earpiece that iswirelessly paired to the mobile device. The drawbacks to this approachinclude the limited battery charge time of the headset or earpiece andthe encumbrance of wearing the ensemble. Another drawback is that theensemble must be kept within relative close proximity to the mobiledevice so as to maintain a wireless link. Also, increasing the distancebetween the mobile device and the ensemble may further drain thebatteries of both the ensemble and the mobile device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a floor plan showing audio units positioned at variouslocations within various rooms and routing audio data in accordance withthe embodiments.

FIG. 2 is diagram showing various views of an audio unit in accordancewith an embodiment.

FIG. 3A and FIG. 3B are schematic block diagrams that provide examplesof various types of audio units that may be used in accordance withvarious embodiments.

FIG. 4 is a schematic block diagram of a mobile device in accordancewith various embodiments.

FIG. 5 is a flow chart of a method of operation of an audio routingsystem in accordance with various embodiments.

FIG. 6 is a flow chart of a method of operation of a master audio unitin accordance with an embodiment.

FIG. 7 is a flow chart of a method of operation of an audio routingsystem in accordance with various embodiments.

FIG. 8 is a flow chart of a method of operation of an audio routingsystem in accordance with various embodiments.

FIG. 9 is a flow chart of a method of operation for handling an incomingphone call by an audio routing system in accordance with variousembodiments.

FIG. 10 is a flow chart providing further details of one operation blockof the flow chart of FIG. 8 in accordance with some embodiments.

FIG. 11 is a flow chart of a method of operation of an audio routingsystem in accordance with some embodiments.

FIG. 12 is a flow chart of a method of operation for a “find me feature”(which is a mobile device location feature) as well as other features ofthe audio routing system in accordance with various embodiments.

DETAILED DESCRIPTION

The disclosed embodiments provide, among other things, an audio routingsystem having features such that a user can transition from room to roomin a house, office, etc., and speak voice commands and hear responseswithout the need for keeping the mobile device in vocal range at alltimes. The audio routing system includes a plurality of audio units andmay include a master audio unit in some embodiments. Each audio unitincludes a speaker and at least one microphone, and plugs directly intoa standard wall AC electrical outlet. This allows the audio unit toreceive electrical power anytime it is needed and overcomes one drawbackof portable electronic devices, namely, discharged batteries. Each audiounit also includes wireless data capability or power-line carrier datacapability and may communicate with other audio units or with a masteraudio unit. The audio units may therefore be installed by plugging theminto electrical outlets throughout a user's home such as in the kitchen,bathrooms, offices, bedrooms, or living rooms of the home.

In one disclosed embodiment, a method of operation includes receivingsound by a first audio unit installed in an electrical outlet, routingaudio data corresponding to the received sound from the first audio unitto a second audio unit installed in a second electrical outlet, andsending the audio data to a mobile device using a wireless link betweenthe mobile device and the second audio unit. In some embodiments,routing the audio data may include receiving the audio data from thefirst audio unit by a third audio unit, and routing the audio data tothe second audio unit by the third audio unit serving as a router.Further, in some embodiments, routing the audio data may include routingthe audio data using one of table drive routing or on-demand routing.

In some embodiments, the method of operation may further includeperforming voice recognition on the audio data and detecting a commandword, and routing command word data to the second audio unit. In someembodiments, routing the audio data to the second audio unit may includesending the command word data from the first audio unit to the thirdaudio unit using a wireless link between the first audio unit and thethird audio unit. In other embodiments, routing the audio data to thesecond audio unit may be accomplished by sending the audio data from thefirst audio unit to the third audio unit using a power line carrierconnection between the first audio unit and the third audio unit.

In another disclosed embodiment, a method of operation includesreceiving sound by a group of audio units installed in a correspondinggroup of electrical outlets. The method further includes selecting afirst audio unit from the group of audio units in response todetermining that the first audio unit meets a threshold with respect toprocessing the received sound. Audio data corresponding to the receivedsound is then routed from the first audio unit installed in a firstelectrical outlet to a second audio unit installed in a secondelectrical outlet, and is sent to a mobile device over a wirelesscommunication link between the second audio unit and the mobile device.

In some embodiments, the method of operation further includes performingvoice recognition on the audio data and detecting a command word, androuting command word data to the second audio unit. Depending on theembodiment, performing voice recognition may involve performing voicerecognition by one of the first audio unit or a master audio unitserving as a router between the first audio unit and the second audiounit. In that case, the audio data may be routed to the second audiounit only in response to detecting the command word.

In some embodiments, selecting a first audio unit may include a masterunit that receives audio data from each audio unit of the group of audiounits, and selects the first audio unit based on comparison of one of anaudio signal level or a voice confidence metric for a command worddetected in the audio data. The metrics are compared for audio datareceived by the master unit from each of the audio units. Depending onthe embodiment, routing audio data may involve routing the audio datafrom the first audio unit to the second audio unit using a wireless linkbetween the first audio unit and the second audio unit. In otherembodiments, routing audio data may involve routing the audio data fromthe first audio unit to the second audio unit using a power line carrierconnection between the first audio unit and the second audio unit overan electrical wiring system connected to a first electrical outlet andthe second electrical outlet.

The disclosed embodiments include an audio unit. The audio unit includesa housing that has a two-prong alternating current power plug with thetwo-prongs extending from the housing, and a transformer and a voltagerectifier installed within the housing and coupled to the power plug.The voltage rectifier is operative to provide direct current to aninternal power bus, which is operatively coupled to the voltagerectifier. The audio unit also includes a microphone, a speaker andaudio signal conditioning logic, operatively coupled to the microphoneand speaker. A first transceiver, operative to establish a connectionwith a second audio unit is operatively coupled to a controller. Thecontroller is operative to control the transceiver to send audio datathe second audio unit. In some embodiments, the audio unit also includesa wireless transceiver that is operatively coupled to the controller,and that is operative to establish a wireless connection with a mobiledevice.

In some embodiments, the audio unit may further include voicerecognition logic. The voice recognition logic is operatively coupled tothe audio signal conditioning logic and to the controller, and isoperative to detect a command word in audio data output by the audiosignal conditioning logic. The controller may be further operative tocontrol the wireless transceiver to send command word data to the mobiledevice, and may be operative to control the first transceiver to sendcommand word data to the second audio unit.

Further, in some embodiments, the controller is also operative tocontrol the wireless transceiver to receive audio data from the mobiledevice, and operative to control the first transceiver to receive audiodata from the second audio unit and to operate the microphone, speakerand audio signal conditioning logic to provide a speakerphone operationwith the mobile device. The first transceiver may be a wirelesstransceiver in some embodiments, or may be a power line carriertransceiver.

Turning now to the drawings wherein like numerals represent likecomponents, FIG. 1 illustrates a floor plan showing an example audiorouting system 100 including various audio units (200A through 200F)positioned at various locations within various rooms in accordance withthe embodiments. Although the example audio routing system 100 is shownon a single level floor plan, the embodiments are not limited to sucharrangements and the audio routing system 100 may cover multiple floorswithin a building. For example, audio unit 200A is installed in anelectrical outlet within room A which is the same room in which mobiledevice 102 is located. The mobile device 102 is paired to the audio unit200A using a wireless link 105. In room B audio unit 200B1 and audiounit 200B2 are installed in respective electrical outlets that arepositioned on walls at a 90° angle with respect to each other. The audiorouting system 100 also includes audio unit 200C, audio unit 200D, audiounit 200E and audio unit 200F which are each installed in correspondingrespective electrical outlets within various rooms. Additional audiounits may also be installed on other floors of a multi-level building.One of the audio units, such as audio unit 200E, may be installed in anelectrical outlet in a relatively centrally located position such as thehallway shown in FIG. 1 and may serve as a “master” audio unit androuter in some embodiments. For example, audio data may be routedbetween audio unit 200B1 and audio unit 200A using the master audio unit200E as a router.

In one example of operation, the audio unit 200B1 will establish acommunication link 107 with the master audio unit 200E via anappropriate routing path, and the master audio unit 200E will establisha second communication link 109 with audio unit 200A via anotherappropriate routing path. These routing paths may be facilitated bypeer-to-peer wireless connections between various audio units, or bypower-line carrier connections over a building electrical system towhich the various AC power outlets are connected.

In accordance with the embodiments, the various audio units located inthe various rooms (an on various floors in some embodiments) serve toextend the audio features and capabilities of the mobile device 102including voice command features. In other words, the user 101 may leavethe mobile device 102 anywhere within a home or office and still be ableto access and control the mobile device 102 using voice commands fromother rooms where audio units are installed. In one example applicationof the audio routing system 100, a user 101 may speak a voice commandwhich may be a query. For example, the user may ask the mobile device102 “Where are you?”

The user's voice creates sound waves 103 and sound waves 104 which maybe picked up by microphones within various audio units. The user's voicecommand query will be routed by one of the audio units within a roomdepending on criteria such as, but not limited to, the “best” receivedaudio signal level, the best voice recognition confidence metric, orsome other suitable criteria. For example, assuming the audio unit 200B1is selected as the best audio unit to transmit the user's voice commandquery, then the audio unit 200B1 will establish a routing path 113 tothe audio unit 200A. As mentioned above, the routing may be facilitatedby a master audio unit 200E in some embodiments. The audio unit 200Awill send audio data, or in some embodiments the command query dataoutput from voice recognition logic, to the mobile device 102 using thewireless link 105. In response to the query, the mobile device 102 maysend audio report data 106 to the audio unit 200A using the wirelesslink 105. The audio unit 200A will then route the audio report data 106to the audio unit 200B1 using an appropriate routing path 113. The audiounit 200B1 will then play an audio report 108 such as, for example, “I'min another room! !” The mobile device 102 may also provide an audioreport 111 from its own speaker such as “I'm here! !” such that the user101 may locate the mobile device 102 as the user 101 moves throughoutthe various rooms. In some embodiments where a master audio unit isemployed, such as master audio unit 200E, the audio unit 200A may usethe communication link 109 to send the audio report data 106 to themaster audio unit 200E for routing on to the audio unit 200B1 usingcommunication link 107. In other words, in some embodiments, the routingpath 113 is facilitated through the master audio unit 200E. However inother embodiments, a master audio unit may not be present and variousad-hoc like routing protocols may be employed such as, but not limitedto, table-driven routing, on-demand routing, or some other suitablerouting protocol.

For example, in the case of table driven routing, a routing table may begenerated by either one of the audio units or by a distributed functiondistributed among the various audio units such that the routing table isstored within each of the various audio units. Thus, if the user were toremove an audio unit from an electrical outlet such that the audio unitis taken out of service from the audio routing system 100, a new routingtable would be generated based on the new audio routing system 100topology and the new routing table would be stored in each of the audiounits remaining in service. In an example of on-demand routing, one ormore audio units receiving sound would broadcast route requests until aroute to the audio unit having a wireless link 105 to the mobile device102 is obtained. In this case, the route may be facilitated directlybetween two audio units or through any suitable number of audio units.For example, in some embodiments, communication link 107 andcommunication link 109 may be wireless links similar to wireless link105. However, in other embodiments, communication link 107 andcommunication link 109 may be facilitated by power-line carriercommunication links over the electrical system wiring to which each ofthe electrical outlets is connected. In the power-line carrier case,direct connectivity may be established between two audio units or datamay be routed using audio units connected along the pathway.

In embodiments where the audio units communicate using wireless links,it is possible that the two audio units that need to communicate areoutside of the wireless coverage range or are otherwise impeded fromcommunicating directly wirelessly. For example, audio unit 200B1 may beunable to establish a wireless link directly with audio unit 200A due toimpedance cause by various walls or other objects such as furniture thatblock the wireless path between the two audio units. Thus if, forexample, audio unit 200D is able to communicate wirelessly with audiounit 200C, and audio unit 200C is able to communicate wirelessly withaudio unit 200B2, then audio unit 200B2 may be used as an initialrouting point for audio data sent wirelessly from audio unit 200B1.Alternatively, audio unit 200B2 may be used as the receiving point forthe user's voice or voice commands based on audio unit 200B2 being ableto establish a wireless routing path to audio unit 200A whereas audiounit 200B1 is unable to communicate with audio unit 200A directly.

FIG. 2 is diagram showing various views of one example of an audio unit200 in accordance with an embodiment. It is to be understood howeverthat the example audio unit 200 is only one possible physicalconfiguration of an audio unit and is not to be construed as alimitation on the particular form that an audio unit may take inaccordance with the various embodiments. Some example features of anaudio unit are shown in FIG. 2 that may be present in audio units havingdifferent physical configurations different than those shown in theexample of FIG. 2 . The audio unit 200 includes a housing 201 which hasan AC power plug 205 installed within the housing 201 and projectingtherefrom such that the audio unit 200 may be installed in a standard ACelectrical outlet such as found in a home or office building etc. Insome embodiments, the prongs of the AC power plug 205 may pivot orrotate such that they may be stored within an indentation (not shown) inthe housing 201 for storing the AC power plug 205 when the audio unit200 is not in use. The housing 201 includes a speaker port 207 and oneor more microphone ports 209. In some embodiments the housing 201includes a status LED 211. A user interface may also be present in someembodiments such as, for example, control button 202 and control button203. The control buttons 202 and 203 may be mechanical pushbutton typecontrols or touch sensitive controls in some embodiments such that thereare no mechanical switch mechanisms involved. The example audio unit 200is light weight and portable and may be similar to an AC charger housingsuch as accessory AC chargers provided with a cord to attach to a mobiledevice for purposes of charging a battery within the mobile device.However as shown in FIG. 2 , the housing 201 does not include anyattached cord with respect to the example audio unit 200.

In some embodiments the control buttons 202 and 203 may be a volumebutton (to adjust response volume) and a mute button. The mute buttonmay silence audio coming from an audio unit or may turn off themicrophone such that the audio unit will no longer receive the user'svoice commands. In other embodiments, the mute button may serve as an“unmute” button if audio is being played from a first audio unit, butthe user desires audio to be played from one or more additional audiounits. For example, a user may speak a command such as “shuffle music”in proximity of a kitchen audio unit. In that case, the mobile devicewould start playing music through the kitchen audio unit. If the userthen moved to a different room, the user could press the mute/unmutebutton on a living room audio unit to playback the music through theliving room audio unit as well.

FIG. 3A and FIG. 3B are schematic block diagrams that provide examplesof various types of audio units that may be used in accordance withvarious embodiments. FIG. 3A illustrates various embodiments of a basicaudio unit 300A and FIG. 3B illustrates various embodiments of a masteraudio unit 300B. Components that may be present in only some embodimentsare shown using dotted lines to emphasize that such components may ormay not be present depending on the particular embodiment. It is to beunderstood that with respect to FIG. 3A and FIG. 3B, like referencenumerals that are reused between FIG. 3 and FIG. 3B indicate like orsimilar components.

Thus with respect to the example basic audio unit 300A and examplemaster audio unit 300B, both include an internal communication bus 327which provides operative coupling between the various internalcomponents. The audio unit 300A or 300B components may include, but arenot limited to, a user interface 307 (which may include control buttons202 and 203), a controller 311A (or controller/router 311B in masteraudio unit 300B), non-volatile, non-transitory memory 312, a wirelesstransceiver 313, a wireless or power-line carrier transceiver 315, andan audio signal conditioner 301. One or more microphones 303 (such as amicrophone array) and a speaker 305 are operatively coupled to the audiosignal conditioner 301. The master audio unit 300B may or may notinclude the audio signal conditioner 301 and the corresponding one ormore microphones 303 and speaker 305. The audio signal conditioner 301,when present in an embodiment, provides audio signal conditioning logicthat may include analog-to-digital converters (ADCs), digital-to-analogconverters (DACs), high-pass filters, low-pass filters, band-passfilters, or adjustable band filters that may be applied to filter noisefrom received audio. The audio signal conditioner 301 may include gainincluding automatic gain control (AGC) in some embodiments. The audiosignal conditioner 301 may also include an echo canceller in someembodiments, to prevent echoing and feedback between the microphones 303and the speaker 305 during full-duplex communication such as when theaudio unit is used as a speakerphone.

A status LED 323 is operatively coupled to switch logic 321 which isfurther operatively coupled to the internal communication bus 327. Theswitch logic 321 is operative to receive control commands from thecontroller 311A (or controller/router 311B) and to light the LED 323 inresponse to the control commands accordingly. In some embodiments, theLED 323 may provide one or more indications and may be a multi-color LEDin some embodiments. For example, the LED may light to show when theaudio unit is paired to another audio unit or to a mobile device. TheLED may also provide an indication that the audio unit is in use. An ACpower plug 319 is connected to a transformer/rectifier 317 which isfurther connected to an internal power bus (not shown) that provides DCpower to the internal components of the audio unit. In embodiments wherethe wireless or power-line carrier transceiver 315 is a power-linetransceiver, the power-line transceiver is operatively coupled to the ACpower plug 319 using appropriate safety and filtering circuitry (notshown) so that audio data including voice command data may be sent andreceived over and electrical system when the AC power plug 319 isinserted into a standard electrical outlet and is thus connected to anelectrical wiring system.

In some embodiments, the audio units 300A or 300B may include voicerecognition logic 309, and may include a motion sensor 325. Inembodiments that include the motion sensor 325, either the controller311A or the controller/router 311B will be operative to obtain data fromthe motion sensor 325 and to determine motion in proximity to therespective audio unit. In embodiments in which voice recognition logic309 is present, the memory 312 may store a library of command words orcommand phrases (i.e. a vocabulary) that are to be recognized by thevoice recognition logic 309. In alternative embodiments, the voicerecognition logic 309 may include an integrated memory that stores thecommand word or command phrase vocabulary.

In some embodiments, the basic audio unit 300A may include wirelesstransceiver 313 which is operative to communicate over wireless link 105with mobile device 102. In such embodiments, any basic audio unit maycommunicate with the mobile device 102 such as audio unit 200A shown inthe example of FIG. 2 . In embodiments where the basic audio unit 300Adoes not include wireless transceiver 313, only the master audio unit300B will be capable of establishing the wireless link 105 with themobile device 102. In these embodiments, the basic audio units 300Acommunicate with each other only, or with the master audio unit 300Bonly, by using wireless or power-line carrier transceiver 315.

The master audio unit 300B may also include a docking connector 331which may be used to connect the mobile device 102 for purposes ofcharging and also, in some embodiments, for providing a data connectionto the mobile device 102 by way of the internal communication bus 327.That is, the docking connector 331 may be operatively coupled to theinternal communication bus 327 and to the power bus (not shown). Is tobe understood that in some embodiments, all of the audio units may beinclude the components of a “master” audio unit 300B where all audiounits include audio signal conditioner 301, one or more microphones 303and a speaker 305. In that case, any audio unit may be designated as amaster audio unit while other audio units function only as basic audiounits or as slave devices. In other words, the basic audio unit 300A mayomit some components such voice recognition logic 309, wirelesstransceiver 313, motion sensor 325 or some combination thereof in orderto reduce the cost of the basic audio unit 300A. Likewise the masteraudio unit 300B may omit the audio signal conditioner 301, microphones303 and speaker 305 in order to reduce the cost of the master audio unit300B. The location of the voice recognition logic 309 may depend onwhether a central routing function is used or not. That is, if a masteraudio unit 300B is used to perform a data routing function, it may alsoinclude the voice recognition logic 309 and may perform voicerecognition on audio data received from the various basic audio units.Alternatively, the basic audio units 300A may include voice recognitionlogic 309, and the master audio unit 300B may omit this component. Insuch embodiments, the basic audio units 300A need only send a dataoutput from the voice recognition logic 309 upon detection of commandwords or command phrases that match the vocabulary stored in memory 312.That is, the basic audio units 300A need not send all their receivedaudio data when each basic audio unit 300A includes voice recognitionlogic 309 which reduces bandwidth requirements. In other embodiments,the basic audio units 300A may listen for audio inputs above a certainvolume threshold and may discard or never send audio data to the masteraudio unit if the volume is below a minimum threshold. In suchembodiments, the basic audio units 300A may, or may not, include voicerecognition logic 309. Based on the above examples, one of ordinaryskill will understand that various configurations are possible for theaudio routing system 100 and for audio unit internal configurations inlight of the above description of example basic audio unit 300A andexample master audio unit 300B.

In the various embodiments, the wireless transceiver 313 is operative toestablish the wireless link 105 with a mobile device 102. The wirelesslink 105 may be any suitable type of radio frequency (RF) communicationlink such as, but not limited to, Bluetooth®, Wi-Fi®, or some othersuitable wireless communication link. The wireless or power-line carriertransceiver 315 is operative to establish communication link 329 withother audio units so that audio data or other data including command andcontrol signals may be routed and sent to the mobile device 102. Inembodiments where wireless or power-line carrier transceiver 315 is awireless transceiver, RF protocols such as Bluetooth®, Wi-Fi®, or someother suitable wireless communication link similar to that used bywireless transceiver 313 may also be employed. The controller 311A andcontroller/router 311B are operative to execute one or more wirelesscommunication protocol stacks and to coordinate communications withother audio units.

A schematic block diagram of a mobile device 102 in accordance withvarious embodiments is shown in FIG. 4 . An example mobile device 102includes an internal communication bus 405 which provides operativecoupling between the various components. The mobile device 102components may include, but are not limited to, a wireless transceiver407 operative to communicate with a network, a wireless transceiver 409operative to communicate peer-to-peer with the audio units, near fieldcommunication logic 411, location detection logic 413, a display 415, auser interface 417, at least one processor 401, a non-volatilenon-transitory memory 403, and a sensor hub 419. The sensor hub 419 isoperatively coupled to a plurality of sensors 420 which may includethermal sensors, proximity sensors, accelerometers, gyroscopic sensors,light sensors, etc.

In accordance with the embodiments, the processor 401 is operative toexecute a voice recognition engine 421. The processor 401 is alsooperative to execute one or more wireless protocol stacks 423(corresponding to wireless transceiver 407 and wireless transceiver 409)and various other applications 425. The voice recognition engine 421 mayinteract and communicate with the wireless protocol stacks 423 via anapplication programming interface API 422 and with the various otherapplications 425 via APIs 424. Each of the various components of themobile device 102 that are operatively coupled to the communication bus405 may also accordingly send information to, or receive informationfrom, the processor 401 and are thus operatively coupled to the voicerecognition engine 421, the wireless protocol stacks 423 and theapplications 425 via the communication bus 405 and via the various APIs.The memory 403 may store voice recognition logic executable code 421C,wireless protocol stack executable code 423C, and application executablecode 425C that may be accessed and executed by the processor 401. Thatis, the processor 401 is operative to access and execute any executablecode stored in memory 403.

Example methods of operation of an audio routing system 100 inaccordance with various embodiments will now be described with respectto the flowcharts of FIG. 5 through FIG. 12 . The flowcharts of FIG. 5through FIG. 12 will be described with respect to the audio routingsystem illustrated in FIG. 1 which is useful for understanding theoperations. Thus beginning with FIG. 5 , the method of operation beginsin operation block 501, and sound is received by an audio unit near theuser such as audio unit 200B1. The audio unit 200B1 then sends audiodata to the master audio unit 200E as shown in operation block 503.Assuming a wireless link can be established between the master audiounit 200E and the mobile device 102, audio data is sent to the mobiledevice 102 over the wireless link from the master audio unit 200E asshown in operation block 505. That is, for embodiments in which themethod of operation of FIG. 5 applies, wireless communication with themobile device 102 is facilitated by the master audio unit 200E.

Turning to FIG. 6 , in operation block 601 command words are detected atthe master audio unit 200E. That is, the master audio unit includes thevoice recognition logic 309. The voice recognition logic 309 outputscommand word data based on one or more detected command words and sendsthe command word data to the mobile device 102 via a wireless link asshown in operation block 603. The method of operation then ends.

In the method of operation illustrated in FIG. 7 , and beginning withoperation block 701, command words may be detected by a basic audio unitsuch as the audio unit 200B1 which thus includes the voice recognitionlogic 309. In operation block 703, the audio unit 200B1 may send commandword data to the mobile device 102 by a wireless link 105 from a secondbasic audio unit such as audio unit 200A which is located near themobile device 102. In operation block 705, audio data may then be routedbetween the first audio unit 200B1 and the second audio unit 200A andalso to and from the mobile device 102. The method of operation thenends. For the method of operation of FIG. 7 , the routing may befacilitated by the master audio unit 200E, or may be facilitated byrouting through various other basic audio units such as audio unit 200C,200D, or 200F, or by a direct routing path 113 between audio unit 200B1and audio unit 200A as was described above previously.

In FIG. 8 , the method of operation begins and in operation block 801voice is detected at one or more audio units. For example as shown inFIG. 1 , the user 101 may speak and generate sound waves 103 which arereceived by audio unit 200B1 and sound waves 104 which are received byaudio unit 200B2. As would be understood, other audio units, includingthose in other rooms such as audio unit 200C and audio unit 200D, mayalso receive sound from the user's voice depending on the direction inwhich the user 101 is speaking and the volume of the user's voice atthat particular time. In operation block 803, the audio routing system100 will determine the audio unit in close proximity to the user. Forexample, audio unit 200B1 may be selected. This operation may beperformed by controller/router 311B in master audio unit 200E bycomparing the audio signal levels received from the various audio unitsand selecting the audio unit having the “loudest” received audio. Oneexample criteria that may be compared is signal-to-noise ratio howeverany suitable criteria may be used. In operation block 805, audio datawill then be routed to the mobile device 102 from the audio unitselected as being in close proximity to the user. In operation block807, audio data will then be sent from the mobile device to that sameaudio unit. The method of operation then ends. Thus for example, ifaudio unit 200B2 is selected as being in close proximity to the user101, then audio data from the mobile device 102 will be routed to theaudio unit 200B2 rather than to the audio unit 200B1 such that the audiounit 200B2 plays audio over its speaker while the audio unit 200B1speaker may be temporarily muted.

FIG. 9 is a flowchart of a method of operation for handling an incomingphone call by an audio routing system 100 in accordance with variousembodiments. Thus in operation block 901, depending on whether a masteraudio unit is employed or not, a notification of an incoming phone callmay be received by audio unit 200A over the wireless link 105. Inoperation block 903, the notification is then broadcast to all otheraudio units. This may be facilitated either by using the master audiounit 200E or by a broadcast notification sent directly from audio unit200A to all other audio units. In operation block 905, all of the audiounits will provide an audible alert of the incoming phone call such as aring tone or some other audible alert. In operation block 907, the user101 may respond to the audible alert by speaking a command word.

Thus in operation block 907, command words may be detected at one ormore audio units such as audio unit 200B1 and audio unit 200B2 whichhappen to be located in the same room as the user 101. In operationblock 909, the audio unit in close proximity to the user will bedetermined and, in operation block 911, voice command word data will berouted to the audio unit in proximity to the mobile device 102 which, inthe example of FIG. 1 , is audio unit 200A. In operation block 913, thephone call may be answered based on the user command, and assuming thatthe user commanded the mobile device 102 to answer the phone call, voicedata may subsequently be routed to and from the mobile device 102 usingthe audio units. The method of operation then ends.

FIG. 10 provides further details of operation block 803 of FIG. 8 whichinvolves determining and selecting an audio unit as being in closeproximity to the user. For example, in operation block 1001, audiolevels received at two or more audio units may be compared. In operationblock 1003, the audio unit having an audio level above a threshold valuemay be selected as being the audio unit in closest proximity to the userand the method of operation then ends as shown. This is only one exampleof selecting an audio unit in closest proximity to the user and it is tobe understood that various other methods may be utilized in the variousembodiments. For example, in embodiments where the audio units includevoice recognition logic 309, a voice recognition confidence metric maybe used and compared to determine which is the “best” audio unit ratherthan determining the audio unit in proximity to the user. Also, it is tobe understood that the audio level may not necessarily be an indicationof actual proximity of an audio unit to the user but may instead providean indication of the “best” audio unit to utilize at a particular pointin time. In embodiments where the audio unit includes a motion sensor325, data from the motion sensor 325 may also be used to considerwhether or not the particular audio unit receiving audio is the “best”audio unit (or is the audio unit in closest proximity to the user) atthat particular time. Also, in some embodiments, the best or closestproximity audio unit may dynamically change as the user moves about.

For example, if the user is speaking on a phone call and is walks out ofroom B, through the hallway, and on to the room in which audio unit 200Dis located, then the audio routing system 100 may switch from audio unit200B1 to audio unit 200E, and then to audio unit 200D, as the closestproximity or best audio unit based on either audio level comparisonreceived by the respective audio unit microphones as the user walksthrough the rooms, by using voice confidence metrics, output of a motionsensor 325, or by using any combination of those information sources inorder to make a decision as to when to switch from one audio unit toanother. In other words, one useful feature and advantage of the variousembodiments is that the user 101 may engage in a phone call while themobile device 102 is located in a remote room, even when the user 101 iswalking from one room to another, provided that there are a number ofaudio units such as audio units 200A through 200F installed in variousrooms and in locations about which the user 101 is traveling andspeaking. As would be understood, the selected audio unit is also usedto route audio data from the mobile device 102 such that the speakeroutput of the audio units will move along with the user 101 as well.

The method of operation illustrated by the flowchart of FIG. 11 providesone example of how this “follow me” feature may be accomplished in thevarious embodiments. In operation block 1101, audio data is received andsent using a first audio unit determined to be in closest proximity tothe user (or the “best” audio unit as discussed above). In operationblock 1103, either the user's voice or user motion is detected by atleast a second audio unit. In decision block 1105, a threshold is testedto see if it is met at a second audio unit. The threshold may be testedby the second audio unit itself, such as by controller 311A, or may betested by a master audio unit such as by controller/router 311B. Themaster audio unit may compare results of one or more potential secondaudio units and select the current closest proximity audio unit. If thebasic audio units perform testing themselves, a predetermined thresholdis used. If the threshold is met, then the method of operation proceedsto decision block 1107. The threshold in decision block 1105 may be athreshold using any of the criteria discussed previously above such as,but not limited to, an audio level threshold, a voice confidence metricthreshold, a motion detection threshold or some other appropriatethreshold. In decision block 1107, a time duration requirement may alsobe tested. If either the threshold of decision block 1105 or the timeduration of decision block 1107 is not met, then the method of operationmay return to operation block 1101. However, if both the threshold andtime duration are met, the method of operation may proceed to operationblock 1109 in which the first (previous) audio unit may be muted andplaced into a low power mode. In operation block 1111, audio data willthen be routed to and from the second audio unit which is now determinedto be in closest proximity to the user or the “best” audio unit.Hysteresis may also be applied with respect to the time duration ofdecision block 1107 to prevent the “popcorn effect” of needless unstableswitching between audio units. The hysteresis may be applied by thecontroller/router 311B of a master audio unit in some embodiments. Uponswitching to the next audio unit, the method of operation then ends.

Thus, applying the method of operation of FIG. 11 , the examplescenarios described above previously where a user may engage in atelephone call and move from one room to another may be facilitated.Other applications may also benefit from the method of operation of FIG.11 , such as audio playback of music or some other type of audio fileplayback from the mobile device 102 over the speakers of the audiounits. For example, the user may listen to music and, as the user movesthroughout the building, the music will follow the user by switchingfrom audio unit speaker to audio unit speaker accordingly.

Turning to the flowchart of FIG. 12 , a “find me feature” (which is amobile device location feature) as well as other features of the audiorouting system 100 are illustrated. Thus, in operation block 1201, audiomay be received by first audio unit in closest proximity to a user (orthe “best” audio unit), for example audio unit 200B1. In decision block1203, the audio may be examined by a voice recognition system todetermine if a voice command is contained in the audio. The voicerecognition logic 309 may be located in any appropriate location and maytherefore be present in either the audio unit 200B1, a master audio unitsuch as master audio unit 200E, or the voice recognition may beperformed by the mobile device 102 depending on the particularembodiment. However, in the embodiments where either the closestproximity audio unit 200B1 or the master audio unit 200E performs voicerecognition, a decision may be made as to whether or not to route theaudio data to a second audio unit, such as audio unit 200A, which is inproximity to, and paired with, the mobile device 102. In other words,audio data may be ignored if it does not contain a voice command asdetermined by the voice recognition logic 309. Therefore, in decisionblock 1203, if a voice command is not contained in the audio, then theaudio unit will continue to receive audio using the first audio unituntil a voice command is detected in decision block 1203. When a voicecommand is detected, the method of operation proceeds to operation block1205 and command word data is routed to a second audio unit. Inoperation block 1207, the command word data from the voice recognitionlogic 309 is then sent to the mobile device 102 using the wireless link105.

If the voice command includes a query as shown in decision block 1209,then, in operation block 1211, the second audio unit 200A will receivequery response audio data, such as audio report data 106, from themobile device 102 over the wireless link 105. In some embodiments, suchas those where a master audio unit 200E is utilized, the query responseor audio report data 106 may be modified to provide additionalinformation if appropriate as shown in operation block 1213. Forexample, based on a routing table or other information known to themaster audio unit 200E, the location of the mobile device 102 withrespect to the user 101 may be known. In that case, thecontroller/router 311B of master audio unit 200E may modify the audioreport data 106 accordingly prior to routing it to the next audio unit.For example, the audio report data may be modified such that the audioreport 108 provided to the user 101 is the message “I'm in another room!!” In other words, the master audio unit 200E may be aware that it iscommunicating with the mobile device 102 via the audio unit 200A whichis located in a different physical position from the audio unit 200B1.In some embodiments, the mobile device 102 location may be triangulatedby the array of home audio units using wireless signals or audiosignals. The master audio unit 200E made thus make an inference that themobile device 102 is located in another room and change the report dataor provide an alternative audio report 108 accordingly. In operationblock 1215, the query response audio data, such as audio report data106, is routed to the audio unit 200B1 so that it will be heard by theuser 101.

Also, in some embodiments, the audio report 111 “I'm here” may be playedby the speaker of the audio unit 200B1 or only by the mobile device 102speaker. However, in such embodiments, the user 101 will have to travelthrough the rooms to figure out where exactly “here” is by listening forthe audio report 111 emanating from the mobile device 102 speakers. Aswould be understood, the audio report 111 may repeat for a predeterminedperiod of time so that the user 101 has time to look around and find themobile device 102.

As was already described with respect to other embodiments, in operationblock 1217, the query response, such as audio report 111 or audio report108, may be played by the audio unit in closest proximity to the user(or the “best” audio unit) using the received audio report data 106 (ormodified audio report data). The method of operation then ends as shown.Returning to decision block 1209, if the voice command does not includea query, then the audio unit in proximity to the user will wait to seeif audio data is received from the mobile device as shown in decisionblock 1219. If not, the method of operation then ends. If yes, then anyaudio data is played using the audio unit in closest proximity to theuser (or “best” audio unit) as shown in operation block 1221 and themethod of operation ends.

In embodiments where a master audio unit is used, other features mayinclude capability to turn off audio units or put them into asleep/standby mode. The master audio unit may also turn audio units offor to standby mode when the mobile device 102 is not in wireless datarange of the master audio unit or the basic audio units depending on theembodiment. For example, in some embodiments, the master audio unit mayturn the basic audio units on or wake them from standby mode when themobile device 102 is in wireless data range of the master audio unit.

Basic audio units may be paired to a master audio unit during aninitialization procedure of the audio routing system 100. For example,audio units may be paired by the user manually, simultaneously pressingand holding a control button on the master audio unit and on the basicaudio unit to initiate pairing. Completed pairing may be indicated bythe status LED 323 illuminating (or showing a certain color such asgreen) or by a sound report from one or both of the master audio unitand the basic audio unit or by both. In other embodiments, the masteraudio unit may include a near field communication (NFC) unit and thebasic audio units may include NFC tags. In such embodiments, pairing maybe accomplished by placing a basic audio unit over or near the masteraudio unit.

In embodiments where the audio units employ Bluetooth®, variousBluetooth® profiles may be used to determine behavior during differentaudio events. For example, during an incoming call, the Hands-FreeProfile (HFP) may be used. This profile supports two-way audio forconversations. In some embodiments, a master audio unit may as a defaultsetting allow the audio ringer to be played on all audio units connectedthroughout the home. A user may then answer via voice command or buttonpress on a specific audio unit or may ignore the call via voice commandor button press. The default setting may be disabled or changed at themaster audio unit if desired by the user either by voice command orcontrol button input.

Some embodiments may include an “answer all” feature in which a call maybe answered and conducted across all available audio units in the home.For example, this feature is useful if a child is calling home fromcollege and the parents are in different rooms of the home and want toparticipate in the call or for other like scenarios.

In some embodiments, for multimedia audio playback on the audio units,the Advanced Audio Distribution Profile (A2DP) may be used. For example,in some embodiments, when the master audio unit receives a multimediaaudio playback signal, the master audio unit may keep basic audio unitsmuted by default. If the master audio unit includes a speaker, it mayplay the audio. Any other audio unit may be unmuted by voice command(such as “unmute”) or by pressing a control button on an audio unit.Thus, a different command to any audio unit or to the master audio unit(such as “unmute all”) may enable playback to all audio units. Anydefault settings may be modified by the user at the master audio unitvia voice command or by using a control button. The above profiledescriptions are examples only, and other profiles or wireless protocolsmay be employed in accordance with the various embodiments.

While various embodiments have been illustrated and described, it is tobe understood that the invention is not so limited. Numerousmodifications, changes, variations, substitutions and equivalents willoccur to those skilled in the art without departing from the scope ofthe present invention as defined by the appended claims.

1. A method implemented by one or more processors, the method comprising: causing audible rendering of music at a first audio unit without any audible rendering of the music at a second audio unit; during the audible rendering of the music at the first audio unit without any audible rendering of the music at the second audio unit: processing, at the second audio unit, audio data that is detected by one or more second unit microphones of the second audio unit and that captures a voice of a user; generating, based on the processing, one or more features of the audio data; comparing the one or more features of the audio data to one or more thresholds; and determining, based on the comparing, to transition the audible rendering of the music from the first audio unit to the second audio unit; and in response to determining to transition the audible rendering of the music from the first audio unit to the second audio unit: initiating the audible rendering of the music at the second audio unit; and ceasing the audible rendering at the first audio unit.
 2. The method of claim 1, wherein the one or more features comprise a voice recognition confidence metric of the audio data; and wherein comparing the one or more features of the audio data to one or more thresholds comprises: comparing the voice recognition confidence metric to a voice recognition confidence metric threshold.
 3. The method of claim 2, wherein the voice recognition confidence metric threshold is a predetermined threshold.
 4. The method of claim 2, wherein the voice recognition confidence metric threshold is based on an additional voice recognition confidence metric generated at a third audio unit based on processing of additional audio data that is detected by one or more additional microphones of the third audio unit.
 5. The method of claim 2, wherein the voice recognition confidence metric threshold is based on an additional voice recognition confidence metric generated at the first audio unit based on processing of additional audio data that is detected by one or more additional microphones of the first audio unit.
 6. The method of claim 1, wherein the one or more features comprise a signal-to-noise ratio of the audio data; and wherein comparing the one or more features of the audio data to one or more thresholds comprises: comparing the signal-to-noise ratio to a signal-to-noise threshold.
 7. The method of claim 1, wherein the one or more features comprise an audio level of the audio data; and wherein comparing the one or more features of the audio data to one or more thresholds comprises: comparing the audio level to an audio level threshold.
 8. The method of claim 1, further comprising: in response to determining to transition the audible rendering of the music from the first audio unit to the second audio unit: placing the first audio unit into a low power mode.
 9. The method of claim 1, further comprising: determining a time duration of the audible rendering of the music at the first audio unit; wherein determining to transition the audible rendering of the music from the first audio unit to the second audio unit is further based on the time duration.
 10. The method of claim 1, wherein causing the audible rendering of the music at the first audio unit, without any audible rendering of the music at a second audio unit, is responsive to a voice command of the user and is responsive to determining that the user was most proximal to the first audio unit when the voice command was provided.
 11. A system, comprising: a first audio unit that includes a first audio unit speaker, a first audio unit microphone, a first audio unit transceiver, first voice recognition logic, and a first audio unit controller; a second audio unit that includes a second audio unit speaker, a second audio unit microphone, a second audio unit transceiver, second voice recognition logic, and a second audio unit controller, wherein the second audio unit controller is configured to: during audible rendering of music via the first audio unit speaker of the first audio unit without any audible rendering of the music at the second audio unit: process audio data that is detected by the second audio unit microphone and that captures a voice of a user; generate, based on the processing, one or more features of the audio data; compare the one or more features of the audio data to one or more thresholds; and determine, based on the comparing, to transition the audible rendering of the music from the first audio unit to the second audio unit; cause, in response to determining to transition the audible rendering of the music from the first audio unit to the second audio unit: initiating of the audible rendering of the music via the second audio unit speaker of the second audio unit.
 12. The system of claim 11, wherein the one or more features comprise an audio level of the audio data; and wherein in comparing the one or more features of the audio data to one or more thresholds the second audio unit controller is to: compare the audio level to an audio level threshold.
 13. The system of claim 12, wherein the audio level threshold is a predetermined threshold.
 14. The system of claim 12, wherein the audio level threshold is based on an additional audio level generated at a third audio unit based on processing of additional audio data that is detected by a third audio unit microphone of the third audio unit.
 15. The system of claim 14, wherein the voice recognition confidence metric threshold is based on a further audio level generated at the first audio unit based on processing of further audio data that is detected by the first audio unit microphone of the first audio unit.
 16. The system of claim 11, wherein the one or more features comprise a voice recognition confidence metric of the audio data; and wherein in comparing the one or more features of the audio data to one or more thresholds the second audio unit controller is to: compare the voice recognition confidence metric to a voice recognition confidence metric threshold.
 17. The system of claim 11, wherein the first audio unit is placed into a low power mode in response to determining to transition the audible rendering of the music from the first audio unit to the second audio unit.
 18. The system of claim 11, wherein the second audio unit controller is further configured to: determine a time duration; wherein in determining to transition the audible rendering of the music from the first audio unit to the second audio unit, the second audio unit controller determines to transition the audible rendering of the music from the first audio unit to the second audio unit further based on the time duration.
 19. The system of claim 11, wherein the first audio unit controller is configured to: cause the audible rendering of the music at the first audio unit responsive to a voice command of the user.
 20. The system of claim 19, wherein causing the audible rendering of the music at the first audio unit is further responsive to determining that the user was most proximal to the first audio unit when the voice command was provided. 